Hi all. I have a small app hosted on Heroku’s free plan, and am looking to migrate away. One feature of my app is that I’d like the ability to execute a script once a week on a set schedule. Heroku has a Scheduler feature which does this for me. And Render has a cron jobs feature. Does Fly.io have any such feature, or allow me to setup something like that?
Thanks in advance.
Hi!
Maybe this will help: New feature: Scheduled machines
Hi thanks very much for that. It sounds like what I’m after. But to be clear, is that a feature I could use, for free, if I setup my app on the Hobby Plan?
And what exactly does that feature do? If I have a single app on the Hobby Plan, the feature allows me to start the app on a set schedule, e.g. monthly, weekly, etc. It would not execute a custom command from the command line (e.g. node myScript.js) on a set schedule. Is that all correct?
Cron is relatively simple if you have access to your Dockerfile (I suppose you do).
First of all, install the cron package by adding the corresponding line(s) to your Dockerfile:
RUN apt-get update
RUN apt-get install -y cron
Once the cron is installed, let’s configure it. Here is a sample crontab file that instructs to run a script every minute:
SHELL=/bin/sh
* * * * * /app/every_1min.sh >> /app/var/log/cron.log 2>&1
Let’s save that file as etc/crontab in your project. Then, we should instruct Docker to copy it to VM by adding the following line to the Dockerfile:
COPY etc/crontab /app/etc/crontab
And here is how our every_1min.sh file looks like:
#!/bin/sh
echo "Hello from cron"
We should copy it to the docker image too:
COPY every_1min.sh /app/every_1min.sh
Now comes the most interesting part. The entry point of the image is represented by CMD directive in Dockerfile. Let’s ensure that we enter our script first so that we can prepare and run our things more naturally.
Here is run.sh script itself:
#!/bin/sh
# Terminate the script on first error.
set -e
# Create directory for log.
mkdir -p /app/var/log
# Load cron configuration.
crontab /app/etc/crontab
# Start cron as a daemon.
cron
# Run your main app.
...
Of course, before we’ll be able to start the script we should copy it first:
COPY run.sh /app/run.sh
And here is CMD directive that starts the script (usually located at the very end of the Dockerfile):
CMD /app/run.sh
You can ensure that cron is working by connecting to VM with
flyctl ssh console
and then looking at /app/var/log/cron.log file. It should have the following lines appended every minute:
Hello from cron
Hello from cron
...
Now you can put whatever command you want to every_1min.sh script and fine-tune the schedule in crontab.
It may be tricky to get it going at first, but once you make it run it becomes endlessly scalable and ready for reuse in other projects.
I’ve created a working sample that shows how to install, configure, and use cron inside a Docker image:
@Hypermind Thanks for all that info. But what if my app doesn’t use Docker?
Mind if I ask why not? I see you mentioned node, and small app. I’m trying to encourage node users to try docker: New nodejs launcher
Could something like the following work for you?
FROM debian:bullseye as builder
ARG NODE_VERSION=18.10.0
ARG YARN_VERSION=1.22.19
RUN apt-get update; apt install -y curl
RUN curl https://get.volta.sh | bash
ENV VOLTA_HOME /root/.volta
ENV PATH /root/.volta/bin:$PATH
RUN volta install node@${NODE_VERSION} yarn@${YARN_VERSION}
#######################################################################
RUN mkdir /app
WORKDIR /app
ENV NODE_ENV production
COPY . .
RUN yarn install && yarn run build
FROM debian:bullseye
COPY --from=builder /root/.volta /root/.volta
COPY --from=builder /app /app
WORKDIR /app
ENV PATH /root/.volta/bin:$PATH
CMD [ "yarn", "run", "start" ]
Change the NODE_VERSION and YARN_VERSION to match node -v and yarn -v. If you aren’t using yarn, remove the setting of YARN_VERSION, installing yarn via volta, and change yarn to npm everywhere. If you don’t have a build step remove && yarn run build. Oh, and remove the [build] instructions from fly.toml.
If your app has special needs that aren’t met by this simple Dockerfile, I’d love to hear about it.
I do appreciate your enthusiasm, and it sounds like you are genuinesly interested in helping. So I’ll give you my honest answer why I’m not interested in trying Docker in this case. Please don’t take any of this as an attack, or attempt to argue with you.
In short, I don’t have any justification for the costs I would incur to try Docker in this case. It would certainly require a fair bit of my time. But what benefit(s) would it provide me? More details below:
Understood. I’m just trying to bridge the gap between the help you received from @Hypermind and where you currently are.
Just some clarifications. You don’t have to install Docker. You don’t have to configure it. It runs on the build machine, and only on the build machine. In fact it doesn’t even run on your deployed host. It is probably best to think of it as a different builder/packager for your application.
Everything but the last RUN and last CMD can largely be ignored.
I’m assuming that you can build and run your application locally on your machine. If you adapt those two lines, fly will reproduce what you do on your machine. The RUN command will be run on the build machine, and the CMD will be run on the deployed machine.
P.S. I neglected to mention that it probably is a good idea to have a .dockerignore file containing something like:
Dockerfile
.dockerignore
node_modules
.git
OK thanks for that clarification. But what would be the benefit to me if I deployed using these steps compared usual steps I’d take to deploy my Node app? If possible, explain in less technical terms?
Again, not trying to challenge or argue with you.
The benefit of a Dockerfile is that if you can describe how you run your application on your machine, those same instructions can be placed in the Dockerfile.
How you currently deploy:
fly deploy
How you would deploy after this change:
fly deploy
In other words, there would be no change. There are three types of builders: Builders and Fly · Fly Docs . By removing a few lines in your fly.toml and creating a Dockerfile you are indicating that you want the instructions in the Dockerfile to be used.
What a .dockerignore file does is list what files you don’t need to upload. The node_modules directory in particular can be quite large and not uploading it can save some time.
Dockerfile approach gives you reproducibility. As @rubys already said, you can run an app created with Docker everywhere. Be it your local machine, Fly, or some other hosting provider.
As a consequence, this gives you the full freedom by breaking proprietary lock-ins and removing a lot of future costs, pains, and hustles.
Another less obvious benefit of using Docker is that you can rely on a cool set of apps and utilities readily available in Linux. Cron is just a small example, but there are literally thousands of other software components that can power up your app. For instance, if your website happens to work with video you can rely on ffmpeg, and so on.
The possibilities are endless and they all are under your fingertips.
OK thanks to both for all that. Now that you describe the reproducibility, I believe I remember reading a description once about how Docker makes it easy for a team of programmers working on an app to create a single development environment for their app, and makes it very fast/easy to bring a new team member on board and get him/her up and running with the exact same dev environment. Does it sound like I’m understanding that part correctly? If so, then yes, as a programmer who has spent many hours trying to get my dev environment to match my team members environment, I can definitely see a big advantage there.
Your understanding is correct - Docker is about reproducibility, among other things.
The classical way of thinking is to gradually tune your machine by changing its state with commands. The problem is that you, as a user, will forget those commands with time. The recipe of reaching the state will be lost in the sands of time. This makes the current state of a system hardly reproducible from scratch.
The Docker way of thinking is to not have a state at all; instead, the list of commands in Dockerfile defines the way of reaching the desired state of a machine in textual form, step by step.
Having a textual form of state gives you extreme powers. For example, you can refine the commands with a text editor, add the file to a version control system, improve it with time, copy & paste to/from other projects, etc.
This allows you to reach the desired state of a machine over and over again, even when it’s starting from scratch. Dockerfile just defines the recipe of reaching that state.
I’ll add to this. I find that sometimes it takes experimentation to find the right “recipe”. And for this I find fly ssh console very useful. I start with a running machine, cleanly installed. I ssh into that machine and try some command that I think will do what I want. If they work, I add them to my dockerfile and redeploy to see if I transcribed them right. If they don’t work, I redeploy without changing my dockerfile and start over with a clean slate, ready for me to try my next experiment.
I find that sometimes it takes experimentation to find the right “recipe”. And for this I find
fly ssh consolevery useful.
I do the same 
I even go a bit further. I always add the following line to my docker files:
# Development
RUN apt-get install -y mc
This gives me an ability to run Midnight Commander right after I log into the machine:
> fly ssh console
# mc
I can then browse and observe the file system more easily with Midnight Commander compared to the usual ls and cd approach.
My second tool of choice is top. It is installed by default and visualizes the consumed resources like CPU load, memory, etc.
Make observations, experiment inside a running VM, make refinements of Dockerfile or app, redeploy. Rinse and repeat until the desired results are reached.
I wrote a guide at Crontab with Supercronic · Fly Docs that shows how to get cron (by way of superchronic) running on Fly.
Hi Brad
I tried to follow the above guide but I get the following error:
Error not enough volumes named app_name_data (1) to run 2 processes
I have the following in my fly.toml file:
`[mounts]
source="app_name_data"
destination="/data"
[[services]]
http_checks = []
internal_port = 8080
processes = ["web"]
protocol = "tcp"
script_checks = []
[services.concurrency]
hard_limit = 25
soft_limit = 20
type = "connections"
[processes]
# The command below is used to launch a Rails server; be sure to
# replace with the command you're using to launch your server.
web = "bin/rails fly:server"
cron = "supercronic /app/crontab"`
I tried autoscaling, but this didn’t work. Any idea why this might be?
I’m not really sure, but it could be that processes of your Fly app, cron and web are both trying to mount the only vol app_name_data onto /data.
Ref:
Don’t forget to update flyctl before deploying the changed config: Multiple processes but only one volume
Each instance requires its own volume. What I think is happening is when your processes launch, it creates two instances of a server, but only one volume is defined. If you’re trying to access one volume from different servers, you can’t.
Assuming you want to mount a volume per process, you’ll need to add this directive to your fly.toml file:
[mounts]
processes = ["app"]
...
We’re not proud of processes yet, and definitely not proud of the lack of documentation. As the v2 stack of apps rolls out that’s based on Machines both volumes and processes will be better architected and documented.