Getting stale `fly logs` on fly instance

Hello. I have an app running in production and generating logs happily. I can see the logs when I run flyctl logs on my laptop and in the dashboard.

I have another app on fly.io, called “logger”, that tails these logs and sends them somewhere. The problem is that when I deployed my production app today, the logger app only got the logs till the “SIGINT” on the last instance, and is not getting the logs from the new instance.

Relevant outputs

On my laptop:

$ flyctl status -a myapp-prod
...
1fb4d4a4	app    	150    	lax   	run    	running	1 total, 1 passing	0       	49m47s ago
$ fly logs -a myapp-prod
...
2022-10-27T22:22:43Z app[1fb4d4a4] lax [info]{"level":30,"time":1666909363549,"pid":680,"hostname":"1fb4d4a4","reqId":"req-ky","req":{"method":"GET","url":"/api/invites?method=health","hostname":"172.22.19.26:8910","remoteAddress":"172.22.19.25","remotePort":52186},"msg":"incoming request"}

On the logger instance

root@e60bddfc:/app# ./flyctl status -a myapp-prod
...
1fb4d4a4	app    	150    	lax   	run    	running	1 total, 1 passing	0       	56m28s ago
root@e60bddfc:/app# ./flyctl logs -a myapp-prod
...
2022-10-27T21:31:17Z runner[93965a77] sjc [info]Shutting down virtual machine
2022-10-27T21:31:17Z app[93965a77] sjc [info]Sending signal SIGINT to main child process w/ PID 528

Sometimes, this same command actually gives me a 404 error.
Does anyone know whey this is happening?

Update: I added another instance to the production app with fly autoscale and the logs from the new instance are showing up, but still nothing from 1fb4d4a4

1 Like