Does a cron manager instance use the primary app's image?

Questions / Help
,
1

Hello,
I am trying to setup cron-manager for my app. I’ve deployed the cron-manager machine with a schedule, but I’m unsure how to use my app’s own built image to launch a job. I have a schedules.json like this:

[
  {
    "enabled": true,
    "name": "my-task",
    "schedule": "0 2 * * *",
    "app_name": "my-app",
    "command": "./my-app jobs run my-task",
    "command_timeout": 900,
    "region": "iad",
    "config": {
      "metadata": {
        "fly_process_group": "cron"
      },
      "auto_destroy": true,
      "image": "my-app",
      "disable_machine_autostart": true,
      "guest": { "cpu_kind": "shared", "cpus": 2, "memory_mb": 512 },
      "restart": { "policy": "no" }
    }
  }
]

but when I attempt to launch the job I get a failure to launch the machine:

root@xxxxxxxxxxxxx:/# cm jobs trigger 1
INFO[0000] Preparing job...                              app-name=my-app job-id=5 schedule=my-task
ERRO[0000] job processing failed                         app-name=my-app error="failed to launch machine: failed to launch VM: failed to get manifest docker-hub-mirror.fly.io/library/my-app:latest: request failed: not found [http 404]: {\"errors\":[{\"code\":\"MANIFEST_UNKNOWN\",\"message\":\"manifest unknown\",\"detail\":\"unknown tag=latest\"}]}\n" job-id=5 my-task panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x798bc2]

goroutine 1 [running]:
github.com/fly-apps/cron-manager/internal/cron.ProcessJob({0xbfbe50, 0x100bae0}, 0xf8a260, 0xc0000401f8, 0x7fddbd81909f26d5?)
        /app/internal/cron/job.go:58 +0x682
main.init.func2(0xf8be00, {0xc000036820?, 0x4?, 0xb3052c?})
        /app/cmd/cm/main.go:114 +0x15b
github.com/spf13/cobra.(*Command).execute(0xf8be00, {0xc0000367f0, 0x1, 0x1})
        /go/pkg/mod/github.com/spf13/cobra@v1.8.0/command.go:983 +0xb34
github.com/spf13/cobra.(*Command).ExecuteC(0xc0000ea008)
        /go/pkg/mod/github.com/spf13/cobra@v1.8.0/command.go:1115 +0x44f
github.com/spf13/cobra.(*Command).Execute(...)
        /go/pkg/mod/github.com/spf13/cobra@v1.8.0/command.go:1039
main.main()
        /app/cmd/cm/main.go:31 +0x1af

I suppose the issue is the image. I want to use my deployed app’s image that I build as part of deployment. How can I reference the latest version of my app’s image?

Thanks for any help!

2

Ah, so you have two apps: one for the scheduler, and one for your main app?

Do you need to run the scheduled jobs inside your main app? Of course, by default they will run on the scheduler, since that is what starts them. Could you expose an API on your app, so the Cron service can reach it? Or you could install the Fly command on the scheduler, and run remote commands on the app.

(I don’t think your crash is specifically relating to not specifying an image; that looks like a implementation bug that should never happen.)

3

Yes, I have two apps in one org: my main app deployment, and a cron-manager deployment (clone of fly-apps/cron-manager with my schedule.json added)

What I’m trying to do (and how I thought I understood that cron-manager is supposed to work) is launch a machine on a schedule that uses my main app’s image, but with a custom command. My app has a job runner command via it’s cli built-in which I want to run on a periodic basis to handle background data tasks.

I could expose an API endpoint, but that just complicates authentication/authorization, etc.

Am I misunderstanding the fundamental method that cron-manager works?

Thanks for your help!

4

Here it’s looking for a public image named my-app, whereas, from your description, you likely instead wanted the private one from registry.fly.io. Try specifying a complete path, including the domain name, etc., matching fly releases --image.

It’s probably wise to double-check your token, as well, since that seems to be a common source of problems in this context.

(I don’t use cron-manager myself, though.)

6

You can, but I think that is not what you currently have. When cron-manager runs, it runs in its own image, which runs permanently. However, you could use flyctl machine exec [machine-id] <command> to run a command from your CM machine on your primary app.

Alternatively, you could install your cron-manager service into the same image as your app, and then use process groups. This would then run the scheduler as a separate machine, but it would have access to your job runner command. It’s import to understand it’s a different machine to your app, but it’s the same image, and thus it would share the same console commands, and the same configuration env vars.

The minor downside with this is your app instances are bloated with a scheduler they don’t need, and your cron instances are bloated with most of the app they don’t need. But in general this is probably not worth worrying about.

7

My app is deployed via the github integration and seems to create image tags with deployment-<commit_sha> with no deterministic tags like latest (unless latest doesn’t show in fly releases –image? I suppose I could test with a known commit, but would like to always use the latest.

I did check my token (and created a new one for this purpose) as i saw that as a common issue in my searching the forum.

8

Right, and it is running its own image. cron-manager itself is deployed and “functional” in that it builds, starts, reads schedules.json, runs migrations against its sqlite db, etc.

When cron-manager tries to run one of the scheduled jobs, my understanding is it uses its $FLY_API_TOKEN to spin up a machine using the app/image in the schedule to execute a command within that spawned machine.

9

Ah, gotcha; that makes sense. I might personally prefer to exec against a running machine, but for short-running jobs, it probably doesn’t matter.

10

I don’t think :latest is supported at the moment. I tried with fly m run, on one of my own apps, and got “not found” errors.

In general, support for mutable tags has been unclear/shakey on registry.fly.io, so, personally, I wouldn’t rely on it.

1 Like
11

I’d suggest reporting this in the relevant repo. This is a low-level Go error that you should never see; there’s probably an access to a freed memory allocation or something similar.

1 Like